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CHAPTER THREE 
THE GREEDY METHOD 
3.1. THE GENERAL METHOD 


Algorithms for optimization problems typically go through a sequence of 
steps, with a set of choices at each step. A greedy algorithm always 
makes the choice that looks best at the moment. 


That is, it makes a locally optimal choice in the hope that this choice 
will lead to a globally optimal solution. A greedy algorithm obtains an 
optimal solution to a problem by making a sequence of choices. At each 
decision point, the algorithm makes choice that seems best at the moment. 
This heuristic strategy does not always produce an optimal solution, but 
as we Saw in the activity-selection problem, sometimes it does. 


Greedy-choice property 

The first key ingredient is the greedy-choice property: we can assemble 
a globally optimal solution by making locally optimal (greedy) choices. 
In other words, when we are considering which choice to make; we make 
the choice that looks best in the current problem, without considering 
results from subproblems. The choice made by a greedy algorithm may 
depend on choices so far, but it cannot depend on any future choices or 
on the solutions to subproblems. 


A problem exhibits optimal substructure if an optimal solution to the 
problem contains within its optimal solutions to subproblems. This 
property is a key ingredient of assessing the applicability of dynamic 
programming as well as greedy algorithms. 


Given problem of n inputs and require us to obtain a subset that satisfies 
some constrains. Any subset that satisfies these constrains is called a 
feasible solution. We need to find a feasible solution that either 
maximizes or minimizes a given objective function. A feasible solution 
that does this is called an optimal solution. 


The greedy method suggests that one can devise an algorithm that works 
in stages, considering one input at a time. At each stage, a decision 
is made regarding whether a particular input is in an optimal solution. 


This is done by considering the inputs in an order determined by some 
selection procedure. If the inclusion of the next input into the 
partially constructed optimal solution will result in an infeasible 
solution, then this input is not assed to the partial solution. 
Otherwise, it is added. 
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The selection procedure itself is based on some optimization measure. 
The measure may be the objective function. In fact, several different 
optimization measures may be plausible for a given problem. Most of 
these, however, will result in algorithms that generate suboptimal 
solutions. This version of the greedy technique is called the subset 
paradigm. 


The function Select selects an input from a[ ] and removes it. The 
selected input ‘s value is assigned to x. Feasible is a Boolean-valued 
function that determines whether x can be included into the solution 
vector. The function Union combines x with the solution and updates the 
objective function. The function Greedy describes the essential way that 
a greedy algorithm will look, once a particular problem is chosen and 
the functions Select, Feasible, and Union are properly implemented. 


Algorithm Greedy (a,n) 
//al[1 : n] contains the n inputs. 


{ 
Solution:=6;// Initialize the solution. 
for i:=1 to n do 
{ 
x:=Select(a) ; 
if Fesible(solution,x) then 
solution := Union(soLution, x); 
} 
} 


* Job Sequencing with deadlines 


Given a set of n jobs, associated with job k is an integer deadline dk 
> @ and a profit pk > @. Constraint is for any job k the profit px earned 
iff (if and only if) the job is completed by its deadlines. To complete 
a job, one has to process the job on a machine for one unit of time. And 
only one machine is available for processing jobs. 


e A feasible solution for this problem is a subset J of jobs such 
that each job in this subset can be completed by its deadlines. 

e The value of feasible solution J is the sum of profits of the jobs 
in J, or Yes Px 

e An optimal solution is a feasible solution that maximize total 
profit. 


2|Page 


e This optimization measure needs us to sort jobs in nonincreasing 
order. So, the objective is to find a sequence jobs, which is 
completed within their deadlines and gives maximum profit. 


E.g., let us consider a set of given jobs as shown in the following 
table. We have to find sequence of jobs, which will be completed within 
their deadlines and will give maximum profit. 


Job J4 J2 J3 J4 Js 
Deadline 2 1 3 2 1 
Profit 60 100 20 40 20 


Job Jo Ji J4 J3 Js 
Deadline 1 2 2 3 1 
Profit 100 60 40 20 20 


From this set of jobs, first we select J2, as it can be completed within 
its deadline and contributes maximum profit. 


e Next select J1, as it gives more profit compared to J4. 
e Next, J4 cannot be selected as its deadline is over, hence J3 is 


selected. 
e Lastly, J5 is discarded as it cannot be executed within its 
deadline. 
Or 
Job Slot Assigned Solution Set (feasible Profit 
Considered solution) 

= = ) 2) 

J [e, 1] {32} 100 

j1 (Gen tite] ‘apie 60 

3 (os a) fit. 2112, 3) {32,915 33} 20 
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Thus, the solution is the sequence of jobs (J2, J1, J3), which are being 
executed within their deadline and gives maximum profit. The total profit 
of this sequence is 100+60+20 = 180. 


Ignoring the sorting step, this job sequencing run in O(n*) time in worst 
cases. 


* Knapsack Problem (fractional knapsack) 


In the knapsack problem, we have n objects and a knapsack or a bag. 
Object k has a weight wx and the knapsack has a capacity m. If a fraction 
Xky @ & Xk £ 1, OF Object k is placed into the knapsack, then a profit 
of pxxXk is earned. (if xx of k placed into knapsack, it will give us 
profit pxXk.) 

e The objective is to obtain a filling of the knapsack that maximizes 
the total profit earned. Since the knapsack capacity is m, we 
require the total weight of all chosen objects to be at most m. 
Formally, the problem can be stated as: 


Maximize Yi<pen PrXk 


Which constrained to \iicpenWeXk < M and O<Xx<1, 1<ksn 


Now, any solution that satisfies both constrain is called feasible 
solution. Likewise, optimal solution is feasible solution that maximize 
the sum of profit. 


Algorithm GreedyKS(m, n) 
{ 
//imagine inputs are already sorted according to the following 
//selection procedures. 
for i:=1 to n do 
x[i] = 0.0; 
k:=1; 
U:=m; 
while k < n do 
if w[k] > U then 
break; 
x[k] = 1.0; 
U:=U-w[k]; 
if k < n then 
x[k]:=U/w[k]; 
} 
E.g., where n = 4 and m = 10. 
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Object (k) 1 2 3 A 
Profit 20 16 24 11 
Weight 7 3 4 6 


There are a least three different measures one can attempt to optimize 
when determining which object to include into the knapsack. Those are 
optimization measure which in turn help us to decide/identify our 
objective function; The total profit, capacity used, and ration of 
accumulated profit to capacity used. 


The selection procedures (order of inputs), or greedy strategies are; 
taking largest profit first, taking smaller weights first, or the ration 
of profit to weight - which is called balanced. In this ratio inputs are 
order in nonincreasing order of their ration, or p1/wl 2 p2/w2 2...2 
pn/wn. 


Now let us order given inputs into the above three selection procedures; 
* Ordering according to nonincreasing profit: 


Object (k) 3 al 2 4 
Profit 24 20 16 11 
Weight 4 7 3 6 


Calculate profit and weight by inserting each input into knapsack, x1, x2, 
x3, x4 are fractions of object k = 1, 2, 3, and 4. 


Following the order: 


[x3, x1, x2, x4] = [1, 3/7, 1, @] 
Mieksa PeXk = 24 + 20*(3/7) + 16 + @ = 48.57 
MickenWeXk = 4+ 3 +3 +0 = 10 


* Order according to nondecreasing order of weights: 


Object (k) 2 3 4 1 
Profit 16 24 11 20 
Weight 3 4 6 7 


Applying the same step: 


[x2, x3, x4, x1] = [1, 1, 1/2, @] 
Mierca PyXk = 16 + 24 + 11*(1/2) + @ = 45.5 
DisksnWeXk = 3 + 4 + 6*(1/2) + @ = 10 
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* Ordering according to nonincreasing order of ratio of profit to weight: 


Object (k) 3 2 1 A 
Profit 24 16 20 11 
Weight 4 3 7 6 


Applying the same step: 


[x3, x2, x1, x4] = [1, 1, 1/2, @] 
Mien<a PyeXk = 24 + 16 + 20*(3/7) + O = 48.57 
DisksnWeXk = 4 4+ 3 + 7* (3/7) + @ = 10 


As you can see from the above; we get the same value for ratio and profit 
base orders; I could be the same, of course. But I didn’t try all possible 
combination to maximize my gains from the fractions of every object. You 
can try it: try to incorporate fraction of last object in the list into 
knapsack. Unlike what I did. 


One thing in general, for knapsack problem you can get less or equal valued 
suboptimal (feasible) solutions: but the ration approach will guarantee you 
the equal or maximum profit. You can rely on it. 


Discarding time for sorting, each of the three strategies require only O(n) 
time. 


3.2. GRAPHS 


* A graph G consists of two sets V and E, G = (V, E). The set V is a 
finite, nonempty set of vertices. The set E is a set of pairs of vertices; 
these pairs are called edges. The notations V(G) and E(G) represents the 
sets of vertices and edges respectively, of graph G. 


* In an undirected graph the pair of vertices representing any edge is 
unordered. Thus, the pair (u, v) and (v, u) represent the edge. In a 
directed graph each edge is represented by a direct pair (u, v); u is 
tail and v is head of the edge. Therefore, (u, v) and (v, u) represent 
two different edges. 


(1) (1) Peet 


a 3 a v3) (2) 
* \ ae f- \ / \ | 

(4 (4 5) (6) (7 (3) 

(a) G, (b) G3 (c) G3 
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The set representations of these graphs are 


V(G,) = {1,2,3,4 E(G 1) = {(1,2), (1,3), (1, 4), (2,3), (2,4), (3, 4)} 
V (Go) = {1,2,3,4,5,6,7} E(G2) = {(1,2), (1,3), (2, 4), (2,5), (3,6), (3, 7)} 
V(G3) = {1,2,3} E(G3) = {(1,2), (2,1), (2,3)} 


If (u, v) is an edge in E(G), then we say vertices u and v are adjacent 
and edge (u, v) is incident on vertices u and v. 


Work out: Find out which vertices are adjacent and List incident edges 
of each vertex. E.g., vertices 1 and 2 are adjacent to each other in 
graph G1, and edges (1, 2) and (1, 3) are incident on vertex 1 in G1 and 
G2. 


* A graph of G is a graph G’ such that V(G’) subset of V(G) and E(G’) 
subset of E(G). A Cycle is a simple path in which the first and last 
vertices are the same. 


3.3. MINIMUM SPANNING TREES 


* Let G= (V, E) be an undirected connected graph. A graph t= (V, E‘) of 
G is a spanning tree of G, iff T is a tree. Spanning trees arises from 
the property that a spanning tree is a minimal subgraph G‘ of G such 
that V(G‘) =V(G) and G‘ is connected. 


* A minimal subgraph is one with fewest numbers of edges. Any connected 
graph with n vertices must have least n-1 edges and all connected graphs 
with n-1 edges are trees. 


In practical situations, the edges have weights assigned to them. These 
weights may represent the cost of construction, the length of the link, 
and so on. Given such weighted graph, one would then wish to select 
cities to have minimum total cost or minimum total length. In either 
case the links selected have to form a tree. The cost of the spanning 
tree is the sum of all the edges in that tree. 


E.g., Look at the following graph a, and its minimum spanning tree b. 
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The two know algorithms used to find minimum spanning tree of a graph 
are Prim’s and Kruskal’s algorithms. 


Finding minimum spanning tree can accomplished by greedy approach, 
because it builds the tree edge by edge. The inclusion of next edge is 
decided based on optimization measure. Simply the optimization measure 
could be “choose an edge that results in a minimum increase in the sum 
of the costs of the edges so far included”. 


KRUSKAL’?S ALGORITHM 


e Actualize the optimization criterion by ordering edges according 
to nondecreasing order of their costs. 

e Let cost[u, v] be weight of edge (u, v). 

e t be a set of edges in the minimum-cost spanning tree. And i be 
the number of edges in t. 

e The set t can be represented as a sequential list using a two- 
dimensional array t[1:n-1, 1:2]. So, the edge (u, v) added to t as 
t[i,1]=u and t[i,2]=v. 

e Let graph G, has n vertices. 


Algorithm Kruskal (E, cost, n, t) 
{ 
Construct a heap out of the edge costs using Happify ; 
for i := n do 
parent [i] := -1 ; 
//now, each vertex is in a different set. 
i := 0 3 mincost := @.@ ; 
while ( i< n- 1 ) and ( heap not empty )) do 
{ 
Delete a minimum cost edge (u, v) from the min-heap and re- 
min-heapify; 
j += Find (u). 3 
k := Find (v) ; 
if (j # k) then 
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i := itl ;5 
€[ 25. 42) toy eft. 2.) tS 3 
mincost := mincost + cost [u, v] ; 


Union (Jj, k); 
} 


if (i # n - 1) then 
Print(“There is no spanning tree’); 
else 
return mincost; 
} 
* j = find(u) returns a root node representing a tree that contain node u. 
Now, the parent of j, parent[j] is going to be -1. Because the parent of 
root is -1 in disjoint set. 
* Union(j, k) - this makes k a parent of j. That means a tree represented 
by root j, is going to be connected to tree k though j. 


Kruskal in simple terms: 
1. Plot all vertices of the graph. 
2. Arrange the edges in nondecreasing order of their weight. 
3. Choose every edge in such a way that it does not form a cycle on the 
graph. 
4. Repeat till all the vertices are connected. 


E.g., given the following graph at figure a, and the minimum spanning tree 
of the graph on figure b (left): 


r) 28 
ee (2 
/ 14? \16 14/ \16 
6) (7) 3 6) (7 3 
2524 \18 Ap 25, 12 
a, ES) 
23 a’ 7274 
(a) ib) 


Steps in kruskal algorithm to find the MST are given as follow; 


9|Page 


Nv 
(Ny 
Nv 


6 7 3 6 7 3 6 ze 3 
12 
5 (s Ss 
4 (4 4 
(a) (b) {c) 
1 ii 1 
2 : 2 2 
10 10 < 10 
14 14 16 14 16 
6 7 3 6 Cz: 3) 6) 7 3 
12 12 12 
5 5 (Ss 
4 (4 22 74 
(d) (Ce) (hy 


Complexity Analysis 

- Construction of min-heap (build-min-heap) takes |E|log|E| time. 

- Making the forest graph or disconnected vertices takes |v| orn 
time. 

- The while loop could go n times, that is equal to |V|, which in turn 
can be taken as |E| (at most). And both find takes 2log|E|, deleting 
min element of min-heap and min-heapify takes log|E| and union takes 
the same, log|E|. Then, 

- [E]loglE] + [E|(loglE| + 2loglE| + logle|) = 5(|E|1log|E|)= 
O(|E|log|E| time. 


3.4. Single-Source Shortest Path 


Let G=(V, E) be a weighted, directed graph with weight function 
w:E->R, and assume that G contains nonnegative weight cycle reachable 
from the source vertex s € V, so that the shortest paths are well defined. 


e The shortest-path tree rooted at s is a directed subgraph 
G’?=(V’,E’), where V’?CV and E’CE, such that 

. VW? is the set of vertices reachable from s in G. 

. G’ forms a rooted tree with root s, and 

For all v € V’, the simple path from s to v in G’ is a shortest 

path from s to v in G. But it is not so necessary for a shortest 

path be unique. 


WN 


Relaxation 


* for each vertex v € V, we maintain an attribute d[v], which is an upper 
bound on the weight of shortest path from source s to v. We call d[v] a 
shortest path estimate. 
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Algorithm Initialize(G, s) 
{ 
for each vertex v € V[G] do 
d[v]:= 


Then, d[v]= © for all v € V-{s}, meaning, except set containing s, all 
other vertices have d[v] of . 


Algorithm Relax(u, v, w) 
4 
if d[v] > d[u] + w(u, v) then 
d[v]:=d[u] + w(u, v) 
} 
Meaning, replace shortest path estimate d[v] with less value, where 
w(u, v) represent weight of edge (u, v). 


Now, we need an algorithm that put all things (initializing, sorting, 
extracting each vertex on min path, repeat the relation for every edges) 
together. 


DIJKSTRA ALGORITHM (DA) 


e DA solves the single-source shortest path problem on a weighted, 
directed graph G=(V, E) for the case in which all the edge weights 
are nonnegative, w(u, v)2@ for edge (u, v) € E. 

e DA maintain a set S of vertices whose final shortest-path weights 
from source s have already been determined. 

e The algorithm repeatedly selects the vertex u € V-S with minimum 
shortest path estimate, adds u to S, and relaxes all edges leaving 
u. 

e We use min-priority queue Q of vertices, keyed by their d values. 


Algorithm Dijkstra (G, w, s) 
{ 
Initialize (G, s) 
S=@ 
Q=V[G] 
While Q # @ do 
u=EXRACT -MIN(Q) 
S=S+{u} 
for each vertex v € Adj[u] do 
Relax (u, v, w) 


Execs 
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Complexity (simplified) 


e |V| number of vertices considered each at a time. 

|E| number of edges relaxed for each v € V, to the worst |V|. This 
may happen when the graph is complete-each vertex connected to 
every other in the graph. 

e By definition, |V|=n. 


Then, T(n) = |V|*|V] = n*n= O(n?). 


Find Chapter Four (Dynamic Programming) from Qubee- 
tp.com 
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